1. Field of the Invention
The present invention relates to a video processing device, a video display device and a video processing method therefor, and a program thereof and, more particularly, to a method of automatically improving quality of a moving image.
2. Description of the Related Art
Image quality improving represents subjecting an original image to image correction processing so as to make still picture and moving picture clearer. Among correction processing for improving image quality are saturation correction and γ (gamma) correction.
Saturation correction is correction intended to adjust saturation indicative of a density of color. Since people are apt to prefer an image whose saturation is high, saturation correction is often conducted so as to adjust saturation of an original image to be high. γ correction is correction intended to adjust brightness of an image. People prefer images of appropriate brightness to images too dark or too bright. Adjusting such brightness is γ correction.
Other than those mentioned above, various kinds of corrections exist and using these correction processing methods to make an image clearer is represented as image quality improving processing. As the above-described image quality improving methods, there conventionally exist such methods as set forth below.
For improving quality of a still image, various still picture quality automatic improving techniques have been used. Still picture quality improving techniques here include those recited in “Color Image Quality Automatic Improvement by Adjustment of Saturation, Contrast, and Sharpness” (Inoue and Tajima, the 24th Image Technology Conference Proceedings 3-3, 1993)(Literature 1), Japanese Patent Laying-Open (Kokai) No. Heisei 09-147098 (Literature 2), Japanese Patent Laying-Open (Kokai) No. Heisei 10-150566 (Literature 3) and “Automatic Color Correction Method Realizing Preferable Color Reproduction” (Tsukada, Funayama, Tajima, Color Forum JAPAN 2000 Proceedings, pp. 9-12, 2000) (Literature 4).
In the automatic image quality improving methods recited in these literatures, a certain feature amount is extracted from an input image composed of still images and a correction amount is determined based on the feature amount to conduct correction for quality improving. Feature amount here represents, for example, an average luminance of a dark region within a screen or an average tone value of each of RGB (R: red, G: green, B: blue) in a bright region within a screen.
In the following, one example of each correction method will be detailed. One example of saturation correction realization methods is shown in FIG. 28. In the present saturation correction realization method, first create a histogram of S values using an HSV (Hue Saturation Value) coordinate system or the like with respect to an input image illustrated in FIG. 28(a) [see FIG. 28(b)]. The HSV coordinate system here is recited in “Color Gamut Transformation Pairs” (A. R. Smith, Computer Graphics, vol. 12, pp. 12-19, 1978).
An S value in the HSV coordinate system denotes saturation, and a histogram of S values therefore can be considered as a histogram of saturation. In the histogram here generated, assume that a high saturation portion where an area ratio to the total number of pixels has a fixed rate “a” is a high saturation region. Then, calculate an average saturation SAF of the high saturation region [see FIG. 28(b)]. Calculate a correction amount Copt from the average saturation SAF according to the following expression:Copt=SAFopt/SAF  (1)
where the average saturation SAFopt represents an optimum value that a saturation image of an input image can take.
The larger thus calculated correction amount Copt becomes, the more saturation will be highlighted [see FIG. 28(c)]. The value of c0 in the figure is that obtained when a range of the saturation S of an input image is expanded to the largest and when c=c0, the saturation S of the input image extends to the largest range as shown in FIG. 28(c) [see FIG. 28(d)].
In image quality improving, obtain an S value from an RGB value of each pixel of a frame image and linearly transform the obtained value according to the following expression:S′=C0pt×S  (2).
After the transformation, restoring the value to an RGB value again leads to completion of an image being corrected. The above-described saturation correction is recited in Literature 1.
One example of exposure correction realization methods is shown in FIG. 29. In the present exposure correction realization method, first create a histogram of Y values using an XYZ coordinate system with respect to an input image illustrated in FIG. 29(a) [see FIG. 29(b)]. Since a Y value denotes luminance, the histogram of Y values can be considered as a luminance histogram.
At this time, with all times the number of pixels set to be m, and with a value of the m-th highest luminance as Zmax and a value of the m-th lowest luminance as Zmin, obtain an intermediate value M of the histogram according to the following expression:M=(Zmax+Zmin)/2  (3).
A γ value with which the intermediate value M becomes M0, half the dynamic range, after the transformation can be obtained by the following expression:γ=[log(255/M0)]/[log(255/M)]  (4).
Exposure correction is realized by first obtaining a Y value from an RGB value of each pixel of a frame image and subjecting the input image to gamma correction by using a γ value obtained by the expression (4) and the following expression with respect to the obtained Y value [see FIGS. 29(c) and (d)].
                    Y        =                              255                          255              γ                                ⁢                      Y            γ                                              (        5        )            
As to the above-described exposure correction, recitation is found in Literature 3.
One example of white balance correction realization methods is shown in FIG. 30. In the present white balance correction method, first create a luminance histogram with respect to an input image illustrated in FIG. 30(a) using an XYZ coordinate system or the like [see FIG. 30(b)].
At this time, with “a” times the number of pixels set to be m, consider a mean value of the respective tone values of pixels having the highest to the m-th highest luminances as a white point of the image. With the white point RGB value denoted as (wr, wg, wb) and a white color RGB value obtained after adjustment as (wr0, Wg0, wb0), obtain white balance correction amounts r, g, b according to the following expressions:r=wr0/wr g=wg0/wg b=wb0/wb  (6).
Based on the above obtained correction amounts and the following expressions, linear transformation of each tone value will realize white balance correction as illustrated in FIG. 30(c):R′=r×R G′=g×G B′=b×B  (7).As to the above-described white balance correction, recitation is found in Literature 2.
One example of contrast correction realization methods is shown in FIG. 31. In the contrast correction realization method, first, create a histogram of Y values, that is, a luminance histogram, with respect to an input image illustrated in FIG. 31(a) using an XYZ coordinate system or the like [see FIG. 31(b)].
At this time, with “a” times the number of pixels set to be m, obtain an average luminance Vmax of pixels having the highest to the m-th highest luminances. Similarly, obtain an average luminance Vmin of pixels having the lowest to the m-th lowest luminances [see FIG. 31(b)].
Based on these values, obtain the following expression which is a straight line passing the coordinates (Vmin, 0), (Vmax, 255):V′=a×V+b  (8).
In the expression, V denotes a luminance Y value of a pixel of an original image and V′ denotes a Y value of the pixel transformed. By linearly transforming a luminance of each pixel using the expression (8) for the inversion into an RGB value realizes contrast highlighting. As to the above-described contrast correction, recitation is found in Literature 1.
One example of sharpness correction realization methods is shown in FIG. 32. In the present sharpness correction method, first subject an input image shown in FIG. 32(a) to a high-pass filter to extract an edge component as illustrated in FIG. 32(b). With ss representing a high-pass filter, E(V) representing an edge region, AE(V) representing an area of an edge region, V representing a luminance and ES0pt representing an optimum sharpness of the image in question, the sharpness correction amount k will be obtained by the following expression:
                    k        =                                                            ES                opi                            ·                                                A                  E                                ⁡                                  (                                      V                    ′                                    )                                                      -                          ∫                                                ∫                                      E                    ⁡                                          (                                              V                        ′                                            )                                                                      ⁢                                                                                                V                      ⊗                      ss                                                                            ⁢                                      ⅆ                    x                                    ⁢                                      ⅆ                    y                                                                                            ∫                                          ∫                                  E                  ⁡                                      (                                          V                      ′                                        )                                                              ⁢                                                                                      V                    ⊗                    ss                    ⊗                    ss                                                                    ⁢                                  ⅆ                  x                                ⁢                                  ⅆ                  y                                                                                        (        9        )            
Using k obtained by the expression (9), the sharpening will be conduced based on the following expression:V′=V+k(V{circle around (x)}ss)  (10)
By inversely transforming an RGB value from V′ obtained by the expression (10), sharpness correction will be realized. As to the above-described sharpness correction, recitation is found in Literature 1.
One example of preferable color correction realization methods is shown in FIG. 33. Preferable color correction is making look of color of an image (representing how the color is perceived by a person, which is also the case in the following description) be more preferable to human eyes by approximating the color of the image to color of the object remembered by a person. Specific processing shown in FIG. 33 as an example is conducted in a manner as follows.
Calculate a hue of each pixel of a frame image shown in FIG. 33(a) to create such a histogram of hues as illustrated in FIG. 33(b). Correct the histogram to make hues related to skin color, color of the sky and green color of plants be those producing more preferable colors by adapting color correction parameters given in advance according to each divisional hue region as shown in FIG. 33(c).
As a result, as illustrated in FIG. 33(d), the image has more preferable colors with only the colors of skin, the sky and green of plants changed. Subjecting the image to such processing realizes preferable color correction. The preferable color correction is intended for obtaining color that one finds preferable when looking only at a corrected image and is conducted based on the contents of know-how accumulated in a data base for a long period of time. As to the above-described preferable color correction, recitation is found in Literature 4.
Using such still picture automatic quality improving techniques as mentioned above realizes quality improving of still picture. Used for improving quality of a moving image is a method of improving quality using a fixed parameter. Fixed parameter is a correction amount parameter fixed to a constant value in order to conduct certain correction for a moving image. Fixed parameters are, for example, as follows.
As shown in FIG. 34, generate images corrected with γ values as various gamma correction parameters in the expression (5) changed and compare the images to obtain an optimum γ value with which the image looks clearer by a subjective evaluation test. At the time of subjecting a moving image to γ correction, when the correction is conducted using an optimum γ value without changing a γ value, the γ value can be considered as a fixed parameter. Technique for improving quality of images using such fixed parameter not only in γ correction but also in various correction processing is employed in moving image quality improving processing.
With the above-described conventional systems in which a correction amount is given by a fixed parameter, however, the moving image quality improving techniques fail to appropriately change a correction amount according to a video source and video shooting conditions.
Moving image has its image quality largely varying depending on its video source and video shooting conditions. In terms of the difference in video sources, a moving image obtained from a DVD (Digital Versatile Disc) deck has high saturation and relatively high contrast, while a moving image shot by an individual person using a home-use digital video camera or the like has low saturation and low contrast as well because of properties of cameras.
In terms of shooting conditions, scenery shot by a digital video camera in cloudy weather and that shot in fine weather will partially differ in saturation and contrast. Quality of a moving image thus varies largely depending on circumstances.
On the other hand, with a correction amount determined by a fixed parameter, while an image taken by a digital video camera is clear, a DVD image might exhibit unnatural look because of too much correction in some cases. Although such a case can be coped with by a method of obtaining a fixed parameter for each video source and manually switching and using the parameters according to each video source, the method is not convenient because it fails to cope with different image qualities caused by different shooting conditions and needs manual switching.